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(N 

H ' Let {Xi,Yi), . . . , {Xn, Yn) be a sample of independent replicates of the random vector {X,Y), where Y is the 

univariate dependent variable and X is the covariate of dimension d. Let m(-) be the conditional expectation 
fvj ' of Y given X and let £ be the related regression error term, so that the regression model is 

pi Y^m{X)+e, 

,S^ I where e is assumed to have mean zero and to be statistically independent of X, and the function m is smooth 

but unknown. Our aim is to investigate the nonparametric estimation of the probability density function of 
the error term e. 
> 

00 

-^ ■ 1 Construction of the estimator 

in 

r^ ' Define 

o 

CN ' Ei = Fj - mj„, i = 1, . . . , n. 



X 



where fhin = min{Xi) is the leave-one out version of the Nadaraya- Watson (1964) kernel estimator oim{Xi), 

Here Ko{-) is a kernel function defined on R"* and 6o = bo{n) is a bandwidth sequence. 
The proposed estimator for the density / of e is 

1 sr^^ ,,. ,. ^ „ / £,: -e 



^■■"'^ <..£;u.(x.<EA-„) |:"-^-^-^°'^Hi:rj- "^ 



where Xq is an inner subset of the support X of the covariatc X, Ki{-) is a univariate kernel function and 
61 — bi(ri) is a bandwidth sequence. 

2 Assumptions 

(Ai) The support X of X is a subset of M. , Xq has a nonempty interior and the closure of Xq is in the 
interior of X . 

(A2) The p.d.f g(-) of the i.i.d. covariates Xi is strictly positive over the closure of Xq, and has continuous 
second order partial derivatives over X . 

(A3) The regression function m(-) has continuous second order palatial derivatives over X. 

(A4) The i.i.d. centered error regression terms £i have finite 6th moments and are independent of the 

covariates Xi. 

(A5) The probability density function /(•) of the Ei 's has bounded continuous second order derivatives over 

M. and satisfies supggjj \hp (e)| < 00, where hp{e) = e^/(e), p G [0,2] and k e {0, 1,2}. 

(Ae) The kernel function Kq is symmetric, continuous overW^ with support contained in [—1/2,1/2]'^ and 

satisfies jKQ(z)dz = 1. 

(A7) The kernel function Ki is symmetric, has a compact support, is three times continuously differentiable 
over M, and satisfies jKi{v)dv = 1, j Kf\v)dv ^0 for 1^1, 2, 3, and JvKf\v)dv = /or £ = 2, 3. 

(As) The bandwidth bo decreases to when n — >■ 00 and satisfies, for d* = supjd + 2, 2d}, nb^ /Inn — !> 00 

and ln(l/feo)/ In(lnn) — > cxd when n — >■ cxd. 

(Ag) The bandwidth bi decreases to and satisfies n^'^^^'b^^^ — > 00 when n — > 00. 



3 Auxiliary results 

Proposition 3.1. Define 

^» = ^W^ ^ MX,)-^(X.))A-o(^ 
where 

'x,-x,: 



^- = i E ^o( — 






Then, under (Ai) — (j49), w;e /lawe, /or a/Z e G K and 6o ond fei go to 0, supj |/3i„| ~ Op(&q) and 

■i 



Y^ An Af ) ^ = Op {bl) (nb\ + (n&i) V2 



Proposition 3.2. Set 



1 (X, g Xq) " 






Xj - Xi 



Then, under {Ai) ~ {Aq), we have, for all e £R. and Bq and bi going to 0, 



Proposition 3.3. Let Var„(-) be the conditional variance given Xi, . . . ,X„, and set 



hi 
Then under (Ai) — (Ag), we have, for all e G M, and &o O'^d bi going to 0, 

Var„ ( X^ Cm ] = Op (nbi + nXh'f) Ut + ^ 



Proposition 3.4. Let Rin = 1 {Xi e Xq) {fhin — m{Xi)) lin, where 



C {i-tfK^^'^ /^ g»-^("^m -m{Xi)) -e \ 



Then, under (Ai) — (Ag), we have, for all e G R, and bo and 61 going to 0, 



Var„ f X] ^™ ) = ^P i^Xbi) (bt + ^ 



4 Intermediate results for the propositions 



The proofs of the propositions are based on the following results. 



Lemma 4.1. Define 



dnix) 



1 " / X ~ T^ 

X! ^^° 1 ~T ) ' 9nix) = IE [dnix)] , X e Xq. 



7ibi — 

^ 2 — 1 



foo 



Then under (Ai) — {A2), (Aq) and (Ag), we have, when Bq goes to 0, 

/ InjiV'"' 

sup \gn{x) - g{x)\ = O (bl) , sup |g„(a;) - 5n(a;)l = Op Uq + ^T 



1/2 



and 



sup 

xeXo 



9n{x) g{x) 



*('s-!i) 



1/2 



Lemma 4.2. Assume that (A4) and (Ag) /loZrf. Then, for any 1 < i ^ j < n, 

{fhin -m{Xi),Si) and {fhjn - m{Xj),ej) 
are independent given Xi, . . . ,Xn, provided that \\Xi — Xj\\ > Cbo, for some C > 0. 
Lemma 4.3. Let £„[•] be the conditional mean given Xi, . . . ,Xn, and assume (Ai) — (Ag). Then, 
sup E„ [1 (X, e Xo) (to„, - m{X,)f] ^ Op (b^ + -^ 

l<i<n \ ^^0 



sup E„ [1 {X, G Xo) im,n - m(X,))'] = Op ( 6^ + J_ 
• - * nog 



l<i<n 



Lemma 4.4. Under (A5) and (A7) we have, for some C > 0, and for any e £ M and p G [0,2], 



/-i"Cr)-/M- 


<Ch, 


/-i"'C ;;)«'/<* 


<Ch, 


/-i"('r)-/M- 


<Ch, 



KfH'-jf)^'mde 



i^fM^)^V(e)rf^ 



kI'^ ( '-^ 1 eP/(e)de 



bi 



Cbl, 


(4.1) 


Cbl, 


(4.2) 


Cbl 


(4.3) 



The proof of all these lemmas is postponed in the appendix. 

5 Proofs of the auxiliary results 
Proof of Proposition 13.11 



Assumption {A4) and Lemma r4.4l - (|4.ip yield 



En 

Var„ 



E/^™^! 



(1) ( Si-e 
hi 



E 



A'r 



(1) /£-e 



61 



E/^^> 



E/^™^! 



(1) /^ £»-e 
61 



< 



E/^-^ 



/vr 



(1) /£-e 



fci 



< C7161 max |/3i„| , 

l<i<n 



< Cnbi max |/3i„| 

l<i<n 



Hence the (conditional) Markov inequality gives 



Y, A;„i^W f %^) = Op (nb\ + (n6i)i/2) max^ |/3, 



hi 



so that the proposition follows if we can prove that 



sup \l5in\ =Op(foo) > 
l<i<n 



as established now. For this, define 



(5.1) 



Q{x) = t{x(.Xo){m{X,)-m{x))KJ^-^], v,^{x) ^ — j ^ (C,(x) - E[0(a:)]) , 



and Vnix) = E[Cj(x)]/&o, so that 



7T-- 1 ^'„t(X^) + l^n(^») 



For maxi<i<„ |i'„(Xi)|, first observe that a second-order Taylor expansion applied successively to g{-) and 
to(-) give, for bo small enough, and for any x, z in X, 



[■m(x + boz) — m(x)] g(x + bgz) 



bom'^'^'>{x)z + —zm^'^\x + 9ibQz)z^ 



g{x) + bog'^^\x)z + -^zg^^\x + e2boz)z'' 



for some 6i ~ 6i{x,boz) and 9-2 = 02{x,boz) in [0, 1]. Therefore, since JzK{z)dz = under {Aj), it follows 
that, by (Ai), {A2) and {A3), 



max |f„(Xi)| < sup |f„(a;)| = sup 
< Cbl 



{m(x + boz) — 7n{x)) Ko{z)g{x + boz)dz 



(5.2) 



Consider now the term niaxi<i<„ \i'i„(Xi)\. The Bernstein inequality (see e.g. Serfling (2002)) and (A4) 
give, for any t > 0, 

)n n „ 

< yP(k.,„(X,)| >i) <y / P(k„(x)| >t\X,^x)g{x)dx 
1=1 4=1 -^ 



< 2ncxp — 



(n-l)t2 



2sup,gA-oVar(0(x)/&„^) + |Mt 



where M is such that sup^^p^.^ \Cj{^)\ ^ M. The definition of Ao given in {A2), (A^), {A7) and the standard 
Taylor expansion yield, for bo small enough. 



sup |Cj(2^)| < C'^Oi sup Var((^j(a;)/foo) < ttj sup / (m(x + 6oz) — 771(0:)) Kf^{z)g{x + boz)dz 



x£Xo x£Xo 

so that, for any i > 0, 
This gives 



"0 ^ 



\2 7,-2^ 



< 



eA'o 






C6| 
6^ 



max |z/™(X,)| > ( , ° ^.^^ i < 2nexp 

i<j<n \{n-l)b^J I 



i^lnn 



C + Ct 



In n 
(n-l)b^ 



1/2 



o(l), 



provided that t is large enough and under (^9). It then follows that 



max \vin{Xi)\ = Or 

l<i<7i 



nb^ 



This order, (j5.2p and Lemma HH show that (|5.1|) is proved, since (6Qhin/(n6o))^/^ = O (fog) under (Ag), and 
that 



Pin — ^ • ^ 



Proof of Proposition 13.21 



Assumption (^4) implies that "Em is independent of Si, and that E„[Sj,i] = 0. This yields 



/ ^ Ein-I^i 



..{1) ( Sj-e 
hi 



= 0. 



(5.3) 



Moreover, observe that 
Var„ 



Z— 1 ^ 



^Var„ 



I^^Af^f^ 



By Lemma r4.4l - (j4.ip and (A4), the first term above gives 






61 j'^^"^i i 5, 



(5.4) 



^Var„ 



^jn-f^l 



(1) /g»~e 



foi 



< ^E„[i]yE 



^r 



(1) /£»-e 



foi 



< 



C&icr^ 



EE 



1(X, gA-o) ./X, -X 



("&^)^ UU ^ 



?m 



i^n^ 



< 



CTifT^ " 1(X, G ^-0)5, 



E 



"0 ,= 



4=1 



5. 



where cr^ = Var(£) and 



gv. 



nbi .^,.^'° 



'2 / -^j ^ -^i 



" i=lj7^J 

For the sum of conditional covariances in (|5.4[) , write 



bo 



(5.5) 



i=l J = i 



EEcov^ 

- EEe« 



&1 



^in^l [ j^ ) T^jnJ^i 



hi 



i=l i=i 



i^iri^in-'^i — ; I ^1 ; 



1 1 ''"IS}^T''°' E i: -« f ^^) -« f ^^ ) « fc.fei . 



i=l i=i 



{nV^Yg.ngjr. 



fc=l ^=1 



60 



where ^ki = Efc^'i ((ej ~e)/6i). Moreover, under (A4), it is seen that for k ^ f, ^[£,kiCej] = when 
Card{i,j, k,i} > 3. Therefore the symmetry of Kq imphes that 



Y. H "^o^" 






bi 



S,„Af' ^],S,„i^«^^^ 



61 



EE 



i(x, e Xo)t{x, e a-q) ^^2 /x^-x, \ ^2 



[nbiYgingjn 



Kr 



W 



eK'l^ 



(1) /e^e 



EE 



i(Xi e A<))i(Xj e Xo) Y^ ^ f Ak- ^i \ ^ I ^k - ^^j \ ]grg2i]g2 



[nh-^Yg^ngju 



E^of^^^lA'o 



fc=l 



&o 



61 



&0 



K\ 



(1) /^ £_e 



Hence from Lemma [01 and Lemma H^(|4.ip . we deduce 



EE^o^" 






^in'^l \ j; ) j^jn^i 



61 y-^""^ V 61 

where 'gin is defined as in (j5.4p and 



Of 






Moreover, using Lemma l4. II and some technical details, it can be shown that 

:-■ = Op(l), 2^ ^j = Op(l), fc = l,2. 

Substituting these orders and (jS.Sp in (j5.4p . yields, for &i small enough. 



Var„ 



i:.,„.i-(^) 



bi , 6t 



= 0p -^ + -i+nfen=Op -±+n6l 



5[J 5g 



&i 



5[J 



Finally, this order, (j5.3p and the Markov inequality give 



/ ^ ^m-f^i 



1=1 



61 



Op 



&o' 



1/2 

nbj ] .D 



Proof of Proposition 13.31 



Observe that Lemma l4~2l yields that Ci„ and Cj„ are independent given Xi, . . . , X„ for some C > such that 
\\X,- Xj\\> Cbo. Therefore 

C?i \ ?i n n 

Y. ^"^ = II V^^'« (^'■") + II 5Z ^ (11^* " ^1 II < ^^") '^O^" (C.n, On) . (5.6) 

1=1 / i=l i=l j = i 

Let Ej,j[-] = E,i[-|Xi, . . . ,Xn,Sk,k ^ i\. Since m„ - TO(Xi) depends only on (Xi,. . .,X„,£fe,fc 7^ z), 



5;] Var„ (C„0 < ^ E„ [C^J = ^ E„ 



i=l 



i=l 



1(X, eAfo)(a,„-m(X,))^E, 



K 



'(2) / £* 



61 



with, using by Lemma r4.41 - (j4.ip . 



E,v 



K\^ 



(2) ^g»-e 



?>i 



= /^^fM^l /W&<C^^i- 



Therefore Lemma iTSl implies that 

n n 

^Var„(C„) < C6i^E„[l(X, GA'o)(TO„-m(X,))^] 
1=1 i=i 

For the sum of the conditional covariances of (|5.6p . the order is derived from the following equalities: 



Y^Y.^ (llx^ - X, II < C60) E„ [c„.] E„ [c,„] = Op (71^5^6?) ( 6;i + 4d 

2—1 J=l ^ ^ 

Y, Y, 1 (11^. - Xj\\ < Cbo) E„ [G„0„] = O, (n26^6l/') (&^ + ;^ 

i=l 3 = 1 ^ 

Indeed, since 61 goes to under (Ag), the equalities above ensure that 

n n 

YY^ di^' - ^^11 < ^^0) co^" (<■»' ^j") 



(5.7) 
(5.8) 



i=l j = i 



Op 



,2ud,7/2 



= Op n^5g5{'^ 6^ + -, 



?ife;^ 



Combining this with the inequahty above and (|5.6p . and applying the (conditional) Markov inequality, gives 
the desired result of the lemma. Hence, it remains to prove (j5.7p and (|5.8p . To this end, note that by Lemma 
l4.4K|4.2p and the Cauchy-Schwartz inequality we have 



s„[C™]| = 



E„ 



1 {X, e Xo) (to„ - m{Xi)f 



Kr 



(2) e^-e 



hi 



K?H'-j^]mde 



X E„ [1 (X, € A-o) (m,„ - m{X,)f] 

1/2 



< Ch'l E„ [1 {X, e Xo) (m,„ - m{X,)y 



uniformly in i, so that (by Lemma 14.31 



sup |E„ [On] E„ [Qn]\ < Cbl sup E„ [1 {X, E Xo) (m,„ - m{X,)f] 



Op (bt) bt 



1 



Therefore, since 



^^1 i\\X, - X,\\ < Cbo) = OrinX), 



i=l j = i 



this gives 



^ ^ 1 (||X, - X, II < Cbo) E„ [C,„] E„ [0„] = Op (nXbt) ( &o + "T 



i=l j = i 



which proves (|5.7p . 

For (j5.8p , let /3i„ and Si„ be as in the statement of Proposition 13.11 and Proposition 13.21 respectively, 
and define and Zin = l(Xi G Xo){fhin — m{Xi))'^. This gives Zin = (/3in + 5]i„)^, so that, for any i ^ j, 



^n [S'in^j7iJ ^7) 



Ari"i 



(2) 



6i 



E,;. 



Zjnl^l 



(2) 



£i - e 
6i 



(5.9) 



where 



E. 



bi 



/-'jn^in 



Kr 



(2) /g»-e 



foi 



2/3,„E, 



J?l-"-'Z?l 



Zjjyj, A "l^ 



(2) 



e.-e\1 


+ E„, 


V2 r^{2) 


/e.-e\1 


fei yJ 


V ^1 A 



(5.10) 



10 



By Lemnia r4.41 - (j4.2[) . the first term above gives 



/3f„]E. 



KY- 



(2) ( e,-e 



bi 



< CbtP^n 



Under (A4), the second term of (|5.10p equals 



2^jn V^ r.- fXk-Xj I j^ 






bo 



SkKl^ 



(2) fe.^e 



bi 



E. 



e^Kl^ 



(2) ( £i~ e 



bi 



nb^gjn "V bo 

Therefore, since Kq is bounded under (Ag), we have (using Lemma [4. 41 - (|4. 2 p 

,(2) /£j-e 



For the last term of (|5.10p . write 



^jn-t^i 



bi 



< 



Cbl\/3,„\ 



nb'^gjn 



Er 



5,2, ,(.)^^^ 






(j^^oSjn) 



^ ^ A-o ("^^L-^^ Ko f^i-^ I E. 



2 A^ A^"^" I f, 

V On 

fc=i e=i ^ " 



n 



i{ Xj e Xq) y^ ^^2 / ^fc - ^^1 1 g. 



(?^&0?jn)^ 



E i<o 



k=i,k=^] 



Xh — Xi 



bo 



bo 



^2^(2) £.-e 



Cfee^-ft^i 



(2) /£i-e 



61 



Since 



E„ 



2.^(2) / £»-e 



E,;, 



< max ■^ sup 

< C6?, 



,'Kfr-^ 



{e \ sup 



^^i 



(2) 



e — e 



(5.11) 



(5.12) 



uniformly in i, this gives 

E, 



y2 ^.(2) 



fol 



< 



Cbmx.eXo) ^ ,.2^^fe-^j 



(n6[5gj„)2 



E ^^'0 



Substituting this, (|ET^ and ([5TT|) in ([STTU]) . yields 

(2) f Si- e 



E„: 



^jn-f^l 



&1 



k=l,k^j 



< CblMn, 



bo 



11 



where 



M„ = sup 






n6[55j„ ('^6o5in)2 ^J[ 



k^3 



Hence from (j5.9|) . the Cauchy-Schwarz incquahty and Lemmas 14.31 14.41 we deduce 

n n 
^ ^ 1 (liX, - Xj\\ < Cbo) |E„ [QnCjn]\ 



i=l 3 = 1 



ZiriKi 



(2) (Sj-e 



bi 



1=1 3 = 1 

n n 

< ChlMr, ^ ^ 1 (||X, - X, II < Cbo) Ey2 [i(x, e A'o)(a„, - m(X,))4] E^^ 

i=l J = i 

/ 1 \ " " 



^r 



(2) /£i-e 



6i 



0/ i=i j = i 



Moreover, using Proposition 13. II (which gives supj |/3i„| = Op(6o)), Lemma ITT] and some technical details, it 
can be shown that 



1 



^^"-^^l^° + ;| + il)-^^(^^ n.5 



Substituting this order in the inequality above, yields (j5.8p and finishes the proof of the proposition. □ 

Proof of Proposition 13.41 

Observe that by Lemma [4.21 we have 

(n \ n n n 

E ^"« = E Varn (Am) + E E 1 (11^* - ^1 II < C'^o) Cov„ (i?„„ i?,„) . (5.13) 

i=l I i=\ i=\ 

Let E.„J-] = E„[-|Xi, . . . ,X„,£fc,/i; =^ z], and write 



i=\ 3 = 1 



E Var„ (i?,„) < E lEn [i?™] = E ^» 



1 (X, e A-o) (m„, - m(X,))'E,„ [zf„] 



12 



with, using (A4), the Cauchy-Schwarz inequahty and Lemnia ini - (|4.3[) . 



(3) /^e - t{min - m{X^)) - e^ 



dt 



< E„ 



(1-t) 



A--^ 



bi 



f{e)de 



dt < Cbi. 



Therefore Lemma 14.31 imphes that 

n 

^Var„(i?„) < Cnbi Slip E„[t{X,€Xo){' 



m,n - TO 



l<i<ri 



(^0)'] 



OAnbMbt + ^ 



For the second term of (j5.13p . write 



1 /2 

|Cov„ {Rin,Rjn)\ < (Var„ (i?j„) Var„ {Rjn)) 



< Cbi snp E„ [t [X, e Xo) (to™ - to(X,))'] - Op(6i) ( ^0 + ;^ ) , 



l<i<i^ 



uniformly in i and j. Hence 

n n 

Y, Y. (11^* - ^jH ^ ^^") I'^o^" (■^-' ^j")i 



1=1 3=1 



^ -. s 3 ri n 



0/ i=l 3 = 1 



Op(6i) 6^ + — i (nX 



n6° 



Finally, this order, (|5.14|) and (|5.13p give, since nbQ diverges under (Ag), 



Var I Y R^n J = Op («&i + n%'},hi) (b^ + -^ 



= Op ri^6;^6i [b, 



o"i; "0 



n6j^ 



.n 



(5.14) 



13 



Appendix: Proof of the intermediate results 

Proof of Lemma 14.11 

First note that by (^7), we have JzKo{z)dz — and jKo{z)dz = 1. Therefore {Ai), {A2) and the second- 
order Taylor expansion, yield, for 60 small enough and any x in Xq, 

1 



l5n(a;) -g{x)\ = 

Koiz) 



bog^^\x)z+^-^zg'^^\x + eb^z)z'^ 



dz 



Ko{z) [g{x + b^z) - g{x)] dz 
0^0{x,boz)e [0,1] 



bl 



zg^''>{x + eboz)z' Ko{z)dz 



<Cbl 



so that 



sup \g^{x) - g{x)\ = O (bl) . 

xGXo 

This gives the first result of the lemma. To prove the second and third results of the lemma, note that it is 
sufficient to show that 



sup |5„(x) - 5„(a;)| = Op — J 



1/2 



since gn(x) is asymptotically bounded away from over Xq and that \g^{x) — g{x)\ = 0(6q) uniformly for x 
in Aq. This follows from Theorem 1 in Einmahl and Mason (2005). □ 

Proof of Lemma 14.21 

Since i^o(0 has a compact support under (Ag), there is a C > such that ||Xi — Xj|| > C60 implies that for 
any integer number k of [1, n], K{){{Xk — Xi)/bo) = if Kq{{Xj — Xfc)/6o) 7^ 0. Let Dj C [1, n] be such that 
an integer number k of [1, n] is in Dj if and only if Ko{{Xj — Xfe)/6o) ^ 0. Abbreviate F{-\Xi, . . . , X„) into 
P„ and assume that ||Xi — Xj|| > Cbo so that Di and Dj have an empty intersection. Note also that taking 
C large enough ensures that i is not in Dj and j is not in Di. It then follows, under (A4) and since Di and 



14 



Dj only depend upon Xi, . . . , X„, 



{rhin — m{Xi),ei) 6 A and (mj„ — m(Xj)^ej) G B 



e, e A 



and 



'E^ei^AO} ("^(^^) - "^(^^■) + ^^) ^0 ((^^ - ^^■)/^o) 






,ej eS 



E.eDA{.} ^^0 ((^fe - ^0/&o) 



e^\ eA 



X P„ 



'E^ei^Ab-} ("^(^^) - "^(^^■) + ^^) ^0 ((^^ - ^J-)/^") 



= P„ {{m^„ - to(X,), £,) e ^) X P„ ((%„ - m(Xj), £j) e B) . 



eB 



This gives the result of Lemma 14. 2[ since both (mm — m{Xi),ei) and (mj„ — m{Xj),ej) are independent 
given Xi,...,X„. D 

Proof of Lemma 14.31 

Let Pin as in the statement of Proposition 13. II and set 



gi: 



nm 



yo 



2 / Xj - Xi 



The proof of the lemma is based on the following bound: 



nhi 



E„ 



1 (X, G -Yo) (m,„ - m(X,))* 



<C 



/5. 



, , i(x, e^o)^: 



fc/2 



, fee {4,6}. 



Indeed, taking successively fc = 4 and fc = 6 in ([1]), we have, by (j5.1[) and Lemma |4?T 



sup E,i 

l<i<n 

sup E,i 
l<i<n 



1 (X, e Xq) (m,„ - m(X,))* 



1 {X, G A-q) (m»n - m{X,)f 



""'^'^'^W? 



Op bf, 



nb^ 



^^l^°^+(4^)-^'(^« + 4 



(1) 



which gives the desired results. Hence it remains to prove ([T]). To this end, let Y,in be as in the statement 
of Proposition 13.21 and observe that l{Xi G Xq) {rhm — m{Xi)) = Pin + Sm. Since j5in depends only on 
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{Xi, . . . , Xn), this gives, for any k G {4, 6}, we have 



E. 



1(X, e Xo) (m„ - m(X,))'' < C/3f„ + CE„ [S 



^k 1 
-'inj 



(2) 



The order of the second term of ([2]) is computed by applying Theorem 2 in Whittle (1960) or the Marcinkiewicz- 
Zygmund inequality (see e.g Chow and Teicher, 2003, p. 386). These inequalities show that for linear form 
L = X]7"=i ^jCj with independent mean-zero random variables ^i, . . . , Cn, it holds that, for any fc > 1, 



eIl'^I <c(fc) 






fc/2 



where C(fc) is a positive real depending only on k. Now, observe that for any integer i £ [^,n\ 






1 {X, e Xq) 



e.jKo 



Xi - X, 



Since under (A4), the cram's, (j = 1, . . . , n), are centered independent variables given Xi, . . . , X„, this yields, 
for any k S {4,6}, 



E„ [Ef„] < CE [e'^-] 



1 [Xi G Afo) ■^^ 2 ' ^1 ^ ^i 



{<?9l ,=1 



Hence, this bound and ([2]) imply that 



E„ 



t{X, & Xq) {fh,n - m{X,)y 



feo 



<c 



fe/2 



< 



C\{x,ex^)ti^ 



{nb^)W2)g:_ 



1(X, G A'o)^, 



k/2 



which proves ([T]), and then completes the proof of the lemma. 



Proof of Lemma 14.41 



Set hp{e) = ePf{e), p G [0, 2]. For the first inequality of (|4.ip . note that under (A5) and (A7), the change of 
variable e = e + biv give, for any £ G {1, 2, 3}, 



K 



4e) 



bi 



ePfie)de 



= bi K^^\vfhp{e + biv)dv 
< 6isup|/ip(e)| I Kf\vfdv 

eGR J 



< Cbi, 



(3) 
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which yields the first inequality of ()4.ip . For the second inequality of (j4.ip . observe that under (A7) we 
have Jk[ '{v)dv — 0. Therefore, since hp{-) has bounded second order derivatives under (A5), the Taylor 
inequality gives 



Ki'' ( ^ ) ^^mde 



= bi 



.{£) 



K\'>{v)[hpie + biv)^hpie)] 



dv 



e& 



< 6?sup|/iW(e)| / \vK[''\v)\dv < Cbj 



which completes the proof of (j4.ip . The first inequalities of (|4.2p and (|4.3p are given by (|21)- The second 
inequalities of (J4.2I) and (|4.3[) are proved simultaneously. For this, note that for any integer £ £ {2, 3}, 

i^f^ ( ^— ^ j /ip(e)de = &i f Kf\v)hp{e + biv)dv. 

By (A7), iiTi has a compact support and satisfies Jk[ '{v)dv ~ and JvKl '{v)dv ~ 0. Hence the second 
order Taylor expansion applied to /ip(-) gives, for some d = 0{e, &i, v) G [0, 1], 



K^/) i i^ ) hp{e)de 



bi I k['\v) 



&i / K['^\v)[hp{e + biv)-hp{e)]dv 

,2„2 



bivhl^He) + ^h'^\e + 9b,v) 



dv 



b\ 

2 

b\ 
< ^ sup I 



i;2xf^(w)/i(2)(e + 6»6iw)dw 



|/^(^)(e)| / v'k['\v) 



dv < Cbf.a 
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